Update the GCM with Chemistry for CF by jeromebarre · Pull Request #896 · GEOS-ESM/swell

jeromebarre · 2026-09-10T14:52:59Z

Description

Generalized meteorology replay source: replaced the hardcoded GEOS FP-only replay config (geosfp_exp / geosfp_path / REPLAY_ANA_EXPID: >>>SWELL_FP_EXP<<<) with a source-agnostic met_replay_exp / met_replay_root question pair and get_met_replay_files() / >>>SWELL_REPLAY_*<<< placeholders in prep_forecast_cf.py, so both GEOS FP and GEOS-IT replay archives work without code changes.
Renamed geos_cf_run_dir → geos_cf_rc_dir across task_questions.py, question_defaults.py, and the suite configs to better reflect that it points to the static RC directory, not a run directory.
Fixed c90 ocean exchange-grid mismatch: corrected OGCM.GRIDNAME / IM_WORLD / JM_WORLD in AGCM_c90.rc (was PE360x180-DE / 360 / 180, now PE2880x1440-DE / 2880 / 1440) to match the pinned BCRSLV tile file, fixing a MAPL_LocStreamMod.F90 GRIDINDEX assert crash at GCMInitialize.
Fixed geoschem_config.yml species list parsing crash: removed malformed #-SPECIES (no-space) commented-out complex-SOA species lines that GEOS-Chem's config reader mis-parsed as a literal species named #, causing a MAPL_Generic.F90 internal-state registration crash.
GEOS-Chem analysis species assembly: write_geoschem_analysis_yml() writes Spc001..Spc<nspecies> blocks for active species (sequentially numbered). Per-species analysis settings files are copied and referenced in GEOSCHEMchem_GridComp.rc.
Fixed HISTORY.rc: removed the grid_label: PE90x540-CF / PE360x2160-CF reference on the output collection that pointed to a grid never defined under GRID_LABELS:, causing a MAPL_HistoryGridComp.F90 assert. Output now stays on the native grid, matching the working reference configuration, with the corresponding dead placeholder logic removed from prep_forecast_cf.py.
Added fvcore_layout.rc and wired it into prepare_namelists()'s copied namelist file list.
Consolidated and cleaned gcm_run_geoscf_c90.j: replaced the resolution-specific script with a single gcm_run_geoscf.j shared across resolutions, driven by new sbatch_nodes / BCRSLV placeholders per resolution.
Removed obsolete tr_import / tr_internal from 3dvar_cf_cycle's clean/reset list and dropped tempo_no2_tropo from both 3dvar_cf and 3dvar_cf_cycle observation lists, keeping only tropomi_s5p_no2_tropo for now with the intent to have OMPS O3 and MLS O3 and TropOMI CO in a next PR. Hofx suite is keeping TEMPO NO2 with the older dates.
Updated runtime/decomposition settings: reduced HEARTBEAT_DT in CAP_c90.rc from 900 to 300, increased the c360 NX / NY / NUM_BACKEND_PES decomposition, and added NUM_BACKEND_PES: 8 for c90.
Updated suite configuration test dates and paths (start_cycle_point, final_cycle_point, geos_cf_install_dir, inc_template) to point to the current development builds and test window. This will affect 3dvar_cf and 3dvar_cf_cycle suites

Dependencies

waiting on GEOS-ESM/repo/pull/<pr_number>
waiting on GEOS-ESM/repo/pull/<pr_number>

Impact

repo
...

jeromebarre · 2026-09-10T15:44:47Z

@mranst the suite configs for 3dvar_cf and 3dvar_cf_cycle have been modified. This makes the CI on this PR fails for 3dvar_cf:
AssertionError: Rendered JEDI config for suite 3dvar_cf, /tmp/tmpr335hwc2/3dvar_cf-config/run/20230805T180000Z/geos_cf/jedi_variational_config.yaml did not match comparison version /home/runner/.local/lib/python3.11/site-packages/swell/test/jedi_configs/jedi_3dvar_cf_config.yaml. Please check the file diffs. If these differences are intentional, new mock test files can be generated using swell utility CreateMockConfigs```

Do I need to this update, if yes tell me how to do this. Or can you do it? Thanks

mranst · 2026-09-10T15:47:55Z

@mranst the suite configs for 3dvar_cf and 3dvar_cf_cycle have been modified. This makes the CI on this PR fails for 3dvar_cf: AssertionError: Rendered JEDI config for suite 3dvar_cf, /tmp/tmpr335hwc2/3dvar_cf-config/run/20230805T180000Z/geos_cf/jedi_variational_config.yaml did not match comparison version /home/runner/.local/lib/python3.11/site-packages/swell/test/jedi_configs/jedi_3dvar_cf_config.yaml. Please check the file diffs. If these differences are intentional, new mock test files can be generated using swell utility CreateMockConfigs```

Do I need to this update, if yes tell me how to do this. Or can you do it? Thanks

You can run swell utility CreateMockConfigs to automatically update the mock configs (if using interactive install), then add them to your pr in a commit. If you're using a static install, add -a <path_to_swell>/src/swell/test/jedi_configs to set the path.

jeromebarre · 2026-09-10T16:11:23Z

@mranst the suite configs for 3dvar_cf and 3dvar_cf_cycle have been modified. This makes the CI on this PR fails for 3dvar_cf: AssertionError: Rendered JEDI config for suite 3dvar_cf, /tmp/tmpr335hwc2/3dvar_cf-config/run/20230805T180000Z/geos_cf/jedi_variational_config.yaml did not match comparison version /home/runner/.local/lib/python3.11/site-packages/swell/test/jedi_configs/jedi_3dvar_cf_config.yaml. Please check the file diffs. If these differences are intentional, new mock test files can be generated using swell utility CreateMockConfigs```
Do I need to this update, if yes tell me how to do this. Or can you do it? Thanks

You can run swell utility CreateMockConfigs to automatically update the mock configs (if using interactive install), then add them to your pr in a commit. If you're using a static install, add -a <path_to_swell>/src/swell/test/jedi_configs to set the path.

I just did it. It removed the observation that it should remove but it didn't changed the dates accordingly.
Dates should change from 2023-08-05T18:00:00Z to 2025-04-02T06:00:00. Not sure that is somehting that matters and needs to be done manually?

mranst · 2026-09-10T16:44:14Z

I just did it. It removed the observation that it should remove but it didn't changed the dates accordingly. Dates should change from 2023-08-05T18:00:00Z to 2025-04-02T06:00:00. Not sure that is somehting that matters and needs to be done manually?

It doesn't exactly matter, as they are generated in a dry-run mode and no checks for obs are performed, so the dates are really just placeholders. But, if you want to change it for consistency, you can make the change here:

swell/src/swell/utilities/scripts/create_mock_configs.py

Lines 19 to 53 in 263429a

    
           defaults_dict = {} 
        
           marine_default_datetime = '20210701T120000Z' 
        
           atmosphere_default_datetime = '20251230T000000Z' 
        
           compo_default_datetime = '20230805T1800Z' 
        
           defaults_dict['3dvar_marine'] = {'datetime': marine_default_datetime, 
        
                                            'model': 'geos_marine', 
        
                                            'executable_type': 'variational'} 
        
           defaults_dict['3dvar_marine_cycle'] = defaults_dict['3dvar_marine'].copy() 
        
           defaults_dict['3dfgat_marine_cycle'] = {'datetime': marine_default_datetime, 
        
                                                   'model': 'geos_marine', 
        
                                                   'executable_type': 'fgat'} 
        
           defaults_dict['3dvar_atmos'] = {'datetime': atmosphere_default_datetime, 
        
                                           'model': 'geos_atmosphere', 
        
                                           'executable_type': 'variational'} 
        
           defaults_dict['3dfgat_atmos'] = {'datetime': atmosphere_default_datetime, 
        
                                            'model': 'geos_atmosphere', 
        
                                            'executable_type': 'variational'} 
        
           defaults_dict['hofx'] = {'datetime': atmosphere_default_datetime, 
        
                                    'model': 'geos_atmosphere', 
        
                                    'executable_type': 'hofx'} 
        
           defaults_dict['hofx_cf'] = {'datetime': '20230805T180000Z', 
        
                                       'model': 'geos_cf', 
        
                                       'executable_type': 'hofx'} 
        
           defaults_dict['3dvar_cf'] = {'datetime': '20230805T180000Z', 
        
                                        'model': 'geos_cf', 
        
                                        'executable_type': 'variational'}

mranst · 2026-09-10T21:44:11Z

Tried the tier1 tests, 3dvar_cf fails in GetBackground/20250402T1200Z, it looks like maybe the experiment swell_test doesn't have entries for that period?

jeromebarre · 2026-09-11T13:57:36Z

Tried the tier1 tests, 3dvar_cf fails in GetBackground/20250402T1200Z, it looks like maybe the experiment swell_test doesn't have entries for that period?

Good catch. It's becasue the backgrounds are generated with the GCM cycling. I will change the date for that suite to only one cycle.

mer-a-o

overall looks great. Thanks @jeromebarre. I was able to run an experiment successfully. I only have a few comments and questions about the changes.

mer-a-o · 2026-09-15T16:11:13Z

+### 'Active' is set programmatically based on the analysis_variables config question.
+
+SpeciesName: NO2
+AnalysisFreq: 6


Can we set this dynamically?

mer-a-o · 2026-09-15T16:14:11Z

+AnalysisMinute: 0
+ForwardLooking: false
+FileTemplate: '>>>SWELL_RUNDIR<<</jedi.inc.%y4%m2%d2.%h2z.nc'
+FileVarName: ozone


I think this should match the variable name in the increment and model O3

mer-a-o · 2026-09-15T16:28:31Z

-       NX: 24
-       NY: 144
+       NX: 60
+       NY: 120


Did you notice speed up with changing NX and NY?

mer-a-o · 2026-09-15T16:28:54Z


 IOSERVER_NODES: 4
-NUM_BACKEND_PES: 6
+NUM_BACKEND_PES: 24


mer-a-o · 2026-09-15T16:33:38Z

@@ -0,0 +1,74 @@
+&fv_core_nml


Why was this file moved to SWELL?

mer-a-o · 2026-09-15T17:09:32Z

-setenv  EXPID   GCv14.0_GCMv1.17_c90b 
-setenv  EXPDIR  >>>SWELL_GEOSRUN<<<
-setenv  HOMDIR  $EXPDIR
+setenv  EXPID   ForecastCF_c360_swell 


If EXPID is not used in the code, better remove it.

mer-a-o · 2026-09-15T16:39:50Z


 @ MODEL_NPES = $NX * $NY

+set NCPUS_PER_NODE = 108


mer-a-o · 2026-09-15T17:19:53Z

 #######################################################################
 #SBATCH --time=00:40:00
-#SBATCH --nodes=8 --ntasks-per-node=108
+#SBATCH --nodes=>>>SWELL_SBATCH_NODES<<< --ntasks-per-node=108


does 108 work for c360?

mranst

Looks good to me, tier tests pass.

mranst · 2026-09-15T17:26:52Z

-        default_value: str = "defer_to_platform"
-        question_name: str = "geosfp_path"
+    class met_replay_root(TaskQuestion):
+        default_value: str = "/home/dao_ops"


Generally I'd suggest keeping filepaths in the platform configs (not that it matters that much since we only support Discover anyway)

jeromebarre added 20 commits August 19, 2026 13:47

consolidate path to the geos-cf spaces

b82d1ea

clean the .j script a bit

d7c46ac

save

2e25638

save

63d6f29

Merge branch 'develop' into feature/update_geos_cf_model

c859565

update .j script

a2dbd69

massive cleaup, flexible replay and single .j

147342e

remove optional rst logic

1a2d7d4

remove tempo from var exps defaults for date change

ca56a46

handle that 'yml' a bit more graciously... XD

289ecee

remove dmget

358abd8

fix wrong fc start timedate

4eeecdf

actually that get rst was correct....

29fc5f9

remove tr rst files

da3245b

fix the openwater stuff for c90

ccfa848

add fvcore layout rc file in prepfc

eccfe75

sst ostia stuff

8308b90

fixes forfc to run and cleanup

10e5cc3

cleanup history rc

f1203c7

fix history rc

842f41a

jeromebarre requested review from Dooruk, mer-a-o and mranst September 10, 2026 14:52

jeromebarre added 2 commits September 10, 2026 11:08

an freq correction

648add9

coding norms paths

0678048

jeromebarre requested review from pawales and viral211 September 10, 2026 15:51

update ci yaml

8f1a3a9

jeromebarre added the compo Atmospheric composition related issues label Sep 10, 2026

Merge branch 'develop' into feature/update_geos_cf_model

3523b7b

change date to one cycle for 3dvar_cf

4d9f15b

jeromebarre mentioned this pull request Sep 14, 2026

Add obs to geos_cf #899

Draft

3 tasks

Merge branch 'develop' into feature/update_geos_cf_model

69b0dc7

mer-a-o reviewed Sep 15, 2026

View reviewed changes

mranst approved these changes Sep 15, 2026

View reviewed changes

Uh oh!

Conversation

jeromebarre commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Dependencies

Impact

Uh oh!

jeromebarre commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mranst commented Sep 10, 2026

Uh oh!

jeromebarre commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mranst commented Sep 10, 2026

Uh oh!

mranst commented Sep 10, 2026

Uh oh!

jeromebarre commented Sep 11, 2026

Uh oh!

mer-a-o left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mranst left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jeromebarre commented Sep 10, 2026 •

edited

Loading

jeromebarre commented Sep 10, 2026 •

edited

Loading

jeromebarre commented Sep 10, 2026 •

edited

Loading